Skip to content

test: harden fixture and generated-client validation - #159

Merged
adamcavendish merged 5 commits into
mainfrom
adamcavendish/harden-validation-pipeline
Oct 20, 2026
Merged

adamcavendish merged 5 commits into
mainfrom
adamcavendish/harden-validation-pipeline

Conversation

@adamcavendish

@adamcavendish adamcavendish commented Oct 20, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Make fixture and generated-client validation fail closed, and keep native consumer builds outside the generator's Rust test suite.

Problem

Nested upstream fixture paths repeated directory prefixes, while missing files silently passed their parser tests. Golden validators could return success without compiling a project. Full local validation omitted native golden builds, and the remaining JVM harness launched Gradle inside Rust tests with global locks that survived interrupted processes.

Activating the real nested fixtures also exposed rejection of valid OpenAPI 3.0 boolean exclusive bounds.

Behavior

Area Resulting contract
Upstream fixtures Resolve nested paths once; reject missing files, read errors, and empty discovery.
Malformed documents Execute explicit parse rejections; file-loading failures cannot satisfy them.
Test boundaries Test discovery in-process; keep native tool orchestration in just/CI.
Golden compilation Reject missing manifests and empty selections; full validation builds all targets.
Build reuse Retain Rust target caches; prepare Python dependencies once per backend.
JVM runtime coverage Use native Gradle consumers with unchanged clients, not Rust builds or custom locks.
Union assertions Prefer specific cases over Any; preserve fallback and tagged payloads.
Toolchains Use stable Rust normally, with explicit and separate MSRV validation.

Compatibility and boundaries

  • OpenAPI 3.0 ObjectSchema.exclusive_minimum and exclusive_maximum now use Option<ExclusiveBound> rather than Option<serde_json::Number>. Boolean values retain both true and false; previously accepted numeric inputs keep their existing lowering behavior. OpenAPI 3.1/3.2 bound handling is unchanged.
  • Fixture registration is shared between build.rs and ordinary in-process Rust tests. No Python tooling scripts or subprocess-driven test harnesses are added, including standalone helpers or meta-tests around CLI/build entrypoints. just/CI directly invoke the native validation tools and consumer projects.
  • Every test_fixture_rejections.txt entry must remain in the final registered fixture set. Missing files and entries removed by skip rules now fail registration instead of silently removing their tests.
  • Handwritten JVM tests, consumer manifests, and dependency lockfiles are tracked; regenerated client source and build artifacts remain ignored. Generator coverage does not include native consumer builds.
  • JVM validation uses JDK 27 while generated libraries retain their Java 17 compilation target. Gson final-field reflection warnings and Kotlin Gradle plugin deprecations remain visible and documented; this change does not suppress warnings or rewrite generated dependencies.

@codecov

codecov Bot commented Oct 20, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

- Accept boolean exclusiveMinimum and exclusiveMaximum flags and retain both
  true and false when lowering property validation into the shared IR.
- Preserve the previously accepted numeric representation through ExclusiveBound
  and document the public ObjectSchema field type change.
- Pin boolean and legacy numeric behavior through the public parser and lowerer.
@adamcavendish
adamcavendish force-pushed the adamcavendish/harden-validation-pipeline branch from 66846f8 to bd10691 Compare October 20, 2026 03:41
- Resolve nested fixture paths exactly once and fail missing files, unreadable
  directories, and empty discovery rather than silently passing parser tests.
- Share fixture registration between the Cargo build script and in-process
  regression tests without launching compilers or test runners from tests.
- Exercise malformed upstream documents as explicit parse rejections and remove
  stale duplicated exclusions without weakening valid-document coverage.
- Reject missing build manifests and empty project selections across the native
  validators instead of accepting successful builds with no compiled projects.
- Preserve Rust dependency caches across fixtures and invocations while honoring
  an explicitly configured Cargo target directory.
- Install Python dependencies and resolve the type checker once per backend,
  retaining visible installation failures and ordinary native tool execution.
- Replace the Rust-driven Gradle harness and global directory locks with native
  Java and Kotlin consumer projects using unmodified CLI-generated dependencies.
- Preserve charset, header, null, and union behavior with ordinary JUnit sources
  and checked-in consumer manifests and dependency lockfiles.
- Assert specific alternatives precede Any and verify complete fallback, tagged,
  adjacent, and external payloads instead of wrapper identity alone.
- Select current stable Rust for ordinary validation while retaining explicit,
  separate MSRV checks and generated libraries' declared compilation targets.
- Run every native golden build and Rust/JVM consumer suite from the full local
  entrypoint and independent CI jobs, with reusable build caches.
- Lint orchestration scripts directly and document in-process fixture coverage
  without Python tooling or subprocess-driven test programs.
@adamcavendish
adamcavendish force-pushed the adamcavendish/harden-validation-pipeline branch from bd10691 to 9866d86 Compare October 20, 2026 08:03
@adamcavendish
adamcavendish merged commit 66ae025 into main Oct 20, 2026
31 checks passed
@adamcavendish
adamcavendish deleted the adamcavendish/harden-validation-pipeline branch October 20, 2026 08:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant